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1 . 


Introduction to the revised review model 


This two-phase proportionate review model is linked to the revised quality framework 
for Scottish FE colleges that was published on the SFEFC and HMIE websites on 
28 May 2004. The previous version of the framework had enjoyed a high level of 
ownership in most colleges and was used extensively to carry out self-evaluations 
which have led to significant quality improvement in the sector. Therefore, while 
there are changes in emphasis within quality elements A5 {Learning and teaching 
process ) and A7 {Learner progress and outcomes) and specific revisions to other 
quality elements, the fundamental structure and content of the quality framework is 
not significantly altered. This should ensure sufficient continuity with the previous 
version to avoid disrupting college self-evaluations. The four-point scale {very good, 
good, fair and unsatisfactory) will be retained. The major aims and objectives of the 
revised model relate primarily to building the capacity of FE colleges to sustain and 
enhance quality, and ensuring public accountability. 

2. Aims and objectives of the proportionate review model 

The model is intended to: 

• support, promote and develop quality enhancement and good practice in colleges 
to provide the best possible experience for the learner; 

• support college arrangements for enhancing quality by providing speedy and 
detailed information for college managers and staff on the quality of the 
programmes and services they provide; 

• provide information to Scottish Ministers and the public on the quality of the 
colleges and their educational programmes; 

• provide an independent evaluation to assist colleges in informing prospective and 
current learners of the quality of the college's educational programmes and 
services; and 

• inform the Council's promotion of quality enhancement, funding allocations and 
policy development and provide assurances of value for money for public funders. 

In realising the above aims, it will provide a breadth and depth of review that is 
differentiated in relation to the level of confidence that SFEFC and HMIE have in a 
college’s arrangements for leadership and quality management in significant aspects 
that impact on the quality of the learners' experience. 

3. Key concepts underpinning the model 

Four key concepts underpin the model. Three of these concepts define the primary 
focus of differentiated review. This focus will be on: 

• Learning and teaching process (A5; previously Teaching and learning 
process). 

• Learner progress and outcomes (A7; previously Student achievement) 

• Leadership and quality management (all of the B quality elements) 


The fourth concept is that the follow-through phase of the review will be 
differentiated. The extent of reviewer engagement will be proportionate to the 
provisional strengths and points for development identified during the first phase. 

3 . 1 Leadership and quality management 
The B quality elements will comprise: 

• Educational leadership, direction and management 

• Access and inclusion 

• Guidance and support 

• Resources and services to support the learner 

• Staff 

• Quality assurance 

• Quality > improvement. 

These quality elements reflect the capacity of the college to support the quality of the 
student experience and improve outcomes as evaluated in findings under the A 
elements (which are primarily about curriculum resources, processes and outcomes). 
A further quality element in the outgoing framework: Arrangements for students with 
learning difficulties and disabilities is no longer a separate entity, and its constituent 
quality indicators have been absorbed by the other B elements listed above. 

3.2 Learner progress and outcomes 

SFEFC and HMIE share the sector’s views that the consideration of outcomes such as 
personal and learning skills and employability, and recognition of “distance travelled” 
in learning are essential for a comprehensive evaluation of learner progress and 
achievement. Consequently, quality element A7: Learner progress and outcomes 
focuses more than in the past on aspects of learner progress and achievement 
additional to attainment in award-bearing programmes. These aspects will be 
evaluated against quality indicator A7. 1 : Learner progress and achievement. 

Learner retention and attainment data, and appropriate information on student 
progression will provide evidence for the evaluation of quality indicator A7.2: 
Learner attainment. 

3.3 Learning and teaching process 

The central and most important process in colleges is learning and all that facilitates it 
including: 

• the relevance of programmes 

• the quality of teaching 

• the quality of learning materials 

• the adequacy of staff, materials, equipment and the physical environment 

• the impact of assessment 


• the effectiveness of guidance and support (which with learning materials often 
takes the place of face-to-face teaching for students learning flexibly, such as 
those undertaking distance or online learning). 

Consequently, in A5: Learning and teaching process, the model focuses more on the 
learning process than previously. 

3.4 Differentiated follow-through phase 

Reviews will be differentiated so that in each college they are proportionate to the 
strengths and points for development in Learning and teaching process and Learner- 
progress and outcomes, and the level of confidence that SFEFC and HMIE have in 
the college’s Leadership and quality management arrangements. Following review 
phase one and full discussion with the college, HMIE will confirm an agenda for the 
follow-through phase. The follow-through phase may be at any point on a continuum 
from Tight touch’ to ‘comprehensive’. 

Differentiated review - the two phases 

In the steady state, HMIE will not visit colleges as part of the review process proper 
until November in each academic year. This will ensure that reviewers can receive 
more informed feedback from learners who will normally have experienced at least 
eight weeks of their programme before being interviewed by reviewers. Transitional 
arrangements will be made for 2004-5, with reviews beginning in January 2005. 

4.1 Pre-review (September-October) 

For colleges being reviewed in that academic year, HMIE will: 

• brief the college principal, chair of Board of Management, managers, and student 
association and learner representatives 

• analyse college documentation, including strategic and operational plans, targets, 
Pis and three-year trends. 

These procedures will help to determine the focus of the first phase of the review. 

The managing inspector (MI) will liaise with the college during this period and the 
college will be invited to provide fully current information prior to the first phase of 
the review. The subject areas to be reviewed will be determined by HMIE in 
discussion with the college to ensure that a representative sample of relevant publicly- 
funded activity is evaluated. 

4.2 Review Phase One (November to March ) 

This phase of the review aims to: 

• evaluate Learning and teaching process and Learner progress and outcomes', 

• evaluate the appropriateness of the curriculum portfolio and the effectiveness of 
Leadership and quality management in terms of educational provision, targets, 
professional competence, self-evaluation, and addressing points for development; 
and 


• inform the scope of the follow-through review phase, which HMIE will confirm 
following full discussion with the college. 

4.3 Follow-through review (about six to eight working weeks later - January to 

May) 

The follow-through review phase will assist the college in preparing an agenda for 

action to improve the quality of the learner experience. It will: 

• be designed to be proportionate to the strengths and points for development 
identified during the phase one review 

• investigate and write up selected good practice associated with some of the key 
strengths identified during the phase one review, and other imiovative or sector- 
leading examples of good practice 

• investigate the causes and extent of any significant weaknesses identified in 
Learning and teaching process, and Learner progress and outcomes during the 
phase one review, drilling down using selected (and occasionally all) A quality 
elements 

• investigate the impact and causes of any significant weaknesses identified in the 
Leadership and quality management elements. 


Review Phase One 

The scope of this phase will be the same for all colleges. A single team of HMIs and 
associate assessors (AAs) will be deployed in two groups (a college review group and 
a subject review group) during the Review Phase One week. A single lay member 
will be deployed across the two review groups. 

5 . 1 Composition and scope of the college review group 

The group will usually comprise external reviewers, typically three HMI (one with the 
role of Managing Inspector) and three AAs. It will address each quality indicator in 
the element Leadership and quality management (the B elements) and quality 
indicator A7.2 in the element Learner progress and outcomes. 

5 .2 Composition and scope of the subject review group 

The group will usually comprise one HMI with the role of Assistant Managing 
Inspector (AMI), and five to eleven reviewers (most of whom will be AAs). It will 
address each quality indicator in the element in Learning and teaching process and 
quality indicator A7. 1 in the element Learner progress and outcomes in up to eight 
subject areas depending on the size of the college and its pattern of provision. The 
group will also cover key specialist areas of provision - core skills, flexible learning 
and special programmes. 

Reviewers will undertake a minimum of six observations of learning in each subject 
and specialist area. Reviewers will not allocate individual grades for these episodes 
of learning but will provide lecturers and other facilitators of learning with individual 
feedback on strengths and points for development. Reviewers’ findings will 


contribute to the overall grades awarded for A5 : Learning and teaching process in 
each subject area. 

5.3 Information required prior to Review Phase One 

The following documentation will be requested directly from the college, to be 
provided no later than four weeks prior to the commencement of review phase one. 

• Principal's context statement 

• List of major and strategic external contacts (eg. employers, key partners, 
community groups) 

• Strategic and operational plans for teaching departments and learner support 
sections/ functions 

• Any recent reports on the college by other external quality bodies 

• College Pis for learner outcomes (including trends analyses) 

• Recent self-evaluations and review reports 

• Organisational charts, timetables and programme information (for example, 
prospectuses) 

• College-nominated examples of good practice. 

HMIE will also obtain documentation from other sources. 

• Completed questionnaires from employers and community groups (as bases for 
subsequent telephone interviews) 

• PI data from SFEFC, based on the college’s FES returns. 

HMIE will also supply an indicative list of further documentation to be made 
available during the review. The college will adapt this list as necessary and provide 
documentation to demonstrate the quality of provision and effectiveness of 
Leadership and quality management. 

5 .4 Obtaining learners ’ perceptions 

The revised review model will use new methodologies to obtain improved feedback 
from learners on progress and achievement of outcomes such as the development of 
personal, learning, core and vocational skills, and the acquisition of skills for 
employability and citizenship. 

5.5 The college-nominated review co-ordinator 

The college should identify a senior manager with a comprehensive strategic 
overview to fulfil the role of review co-ordinator. This senior manager will play a key 
role in supporting the smooth operation of the review. Before the review HMIE will 
discuss and agree with each college the proposed nature and levels of reviewer 
engagement with managers and staff. HMI E will encourage college senior managers 
to work in partnership with reviewers to identify appropriate evidence. The role of 
review co-ordinator is to: 


• act as the primary contact and liaison between the HMI E review team and the 
college 

• clarify managers’, staff and team roles and responsibilities 

• direct reviewers to sources of evidence across the college, including the 
appropriate staff to interview on specific issues. 

The college review co-ordinator will not: 

• be expected to convey messages from other managers or staff to members of the 
review team 

• have access to evidence gathered from review team interviews with college 
managers and staff 

• be party to review team discussions or deliberations, other than on the scheduling 
and structure of review activity 

• have any role in determining grades or communicating review team evaluations to 
other managers or staff. 

5.6 Review Phase One: Deployment of the college review group 

The following is indicative rather than prescriptive. It is intended that reviewers’ 
interviews with managers and staff will be targeted at confirming potential strengths 
and points for development provisionally identified through reading college 
documents and discussions with college senior managers. 

From Monday morning to Tuesday lunch time, the group will evaluate documentary 
evidence against all Leadership and quality management elements and quality 
indicator A7.2: Learner attainment in award-bearing programmes and identify 
potential key strengths and points for development. On Monday afternoon, the group 
will be briefed by the principal and senior management team including the review co- 
ordinator. They will also be briefed by the MI. 

During the period from Tuesday afternoon to Thursday morning, the group will 
conduct interviews and undertake other activity requiring to determine the extent of 
concordance between documentary evidence and practice, and evaluate provision to 
identify provisional strengths and points for development. On Thursday afternoon the 
group will work with the subject review group to prepare key findings. On Friday the 
group will give oral feedback to the college, with grades for each quality element 
relating to Leadership and quality > management. 

5.7 Review Phase One: Deployment of the subject review group 

From Monday lunch time to Tuesday lunch time the group will evaluate documentary 
evidence relating to quality element A5: Learning and teaching process and quality 
indicator A7.2: Learner attainment in award-bearing programmes and identify 
provisional key strengths and points for development. (In considering A7.2, the 
group will build on the work done by the college review group). On Monday 
afternoon, the group will be briefed jointly with the college review group by the 
principal, the senior management team including the review co-ordinator, and the MI. 
They will also be briefed by the AMI on Tuesday morning. 


From Tuesday afternoon to Thursday morning the group will: evaluate all quality 
indicators in the element A5: Learning and teaching process', evaluate quality 
indicator A7. 1 : Learner progress and achievement', observe episodes of learning and 
teaching; and provide feedback to each lecturer or other learning facilitator on 
strengths and points for development. On Thursday afternoon the group will prepare 
key findings working with the college review group. On Friday the group will give 
oral feedback to the college with grades for quality elements A5: Learning and 
teaching process and A7: Learner progress and outcomes. The group’s feedback on 
the latter quality element will be based on its own evidence gathered at first hand on 
A7.1 : Learner progress and achievement, together with its views on A7.2: Learner 
attainment in award-bearing programmes arrived at in discussion with the college 
review group. Separate feedback will be given to operational managers and staff, and 
to the principal and senior managers. 

The subject review group will also share with the college the extent to which it 
considers that any major strengths or significant weaknesses in A5: Learning and 
teaching process and A7: Learner progress and outcomes may relate to good practice 
or weaknesses in the other A elements: Programme design, Accommodation for- 
teaching and learning, Equipment and materials, Staff, Assessment, Guidance and 
learner support, and Quality assurance and improvement. While these elements will 
not be evaluated directly during review phase one, reviewers evaluating against A5 
and A7 will form provisional views on the: 

• relevance of programmes to learners' needs 

• adequacy of the physical learning environment 

• adequacy of materials and equipment to support learning and teaching 

• adequacy of staff to deliver and support learning and teaching 

• impact of assessment arrangements on the learner 

• adequacy of guidance and support 

• impact of quality assurance and improvement arrangements on the quality of the 
learners’ experience. 

These provisional views will be expressed in terms of identified strengths and 
weaknesses. They will help to inform any subsequent ‘drilling down’ using all A 
elements and, where appropriate, the B elements, during the follow-through phase. 


5 . 8 Typical reviewer deployment in Phase One 


PHASE ONE REVIEW TEAM 

DEPLOYMENT 


College review group 

6 Reviewers (3 HMIs, 3 AAs) 
1 HMI = MI 

Subject review group 

5-11 Reviewers (mainly AAs) + LM 
(working across both groups) 

+ HMI = AMI 

Monday 



(am) 

(pm) 

Evaluate documentary evidence for B 

elements 

Evaluate A7.2 

Briefings by principal/senior 
management team and MI 
Identify provisional key strengths and 
points for development 

Briefings by principal/ senior 
management team and MI 
Evaluate documentary evidence 

Tuesday 



(am) 

Evaluate documentary evidence 
Identify key strengths and points for 
development 

AMI briefing on A7.2 
Evaluate documentary evidence 

(pm) 

Conduct interviews 

Evaluate A5 and A7.1 

Wednesday 



(am) 

Conduct interviews 

Evaluate A5 and A7.1 

(pm) 

Conduct interviews 

Evaluate A5 and A7.1 

Thursday 



(am) 

Conduct interviews 

Evaluate A5 and A7.1 

(pm) 

Prepare key findings with MI and 
subject review group 

Prepare key findings with AMI and 
college review group 

Friday 



(am) 

Feedback to college managers on all 
B elements (9.30- 12.00) 

Feedback to subject managers on A5 
and A7 (9.30- 12.00) 

(pm) 

Feedback to senior managers (12.30 - 
4.00) 

Preliminary indication of the scope and n 

Feedback to senior managers 
(12.30-4.00) 

ature of the follow-through 


I 


(6-8 weeks later) 

Proportionate follow-through activities 


5.9 Statements of confidence 
Provisional statements of confidence 

At the end of the phase one review HMIE will, in addition to awarding a grade for 
each B quality element and for A5 and A7, express provisional statements of 
confidence in the college. These statements may be applied to the college as a whole 
but in some circumstances HMI E may express different levels of confidence in the 
three aspects of Learning and teaching process'. Learner progress and outcomes', and 
Leadership and quality management. Provisional confidence statements will be 
confidential to HMIE, SFEFC and the college principal. At the request of the 
principal HMIE will share provisional confidence statements with other college staff. 

The format of confidence statements will be based on the following: 

• HMIE is provisionally confident that .... College has effective learning and 

teaching processes and/or, learners are achieving appropriate outcomes, and/or the 
college is managing well and improving the quality of its services for learners [but 
{summary of any concerns).] 


AND/OR 

• HMIE is provisionally not confident that .... College has effective learning and 
teaching processes and/or learners are achieving appropriate outcomes and/or the 
college is managing well and improving the quality of its services for learners. 
HMIE is concerned that {summary of major concerns). 

Confidence statements will be based on colleges meeting the following seven 
confidence criteria. Indicative elements for follow-through review most likely to be 
used are not bracketed but those in brackets may also be used. 


Confidence criteria 

Elements used 
in review 
phase one 

Indicative elements 
for follow-through 
review 

1 . The learning and teaching process is good or very good in the 
subject and specialist areas evaluated. 

A5 

A8 (A5, A7) 

2. Learner progress and outcomes are good or very good in all of 
the subject and specialist areas reviewed and learner retention 
and attainment is high across almost all subject areas. 

A7 

(A1 - A9) 

3. Educational provision is well designed, planned and managed, 
and responsive to national, community, learner, employer, 
economy and stakeholder needs. 

B1,B2, B3, 
B4 

A1 (A2 - A9) 

4. Demanding but realistic targets and key performance 
indicators (KPIs) are being set and largely achieved. 

Bl, B7.2 

A9 (A1 - A8) 

5. Levels of professional competence are high and being 
enhanced for all staff. 

B5 

A4 

6. Programmes, resources and all services for learners are 
evaluated systematically, points for development are 
addressed and improvements made; and the quality of learning 
and teaching is evaluated and improved. 

B6, B7 

A9 

7. Any major points for development have already been 
identified and acted on by the college. 

B1-B7 

A1-A9 


(Criteria 6 and 7 include the validation of the effectiveness of the college’s 
self-evaluation processes) 

Final statements of confidence 

The provisional confidence statement after phase one review will be finalised 
following the follow-through review, as follows: 


• HMIE is confident that .... College has effective learning and teaching processes, 
and/or learners are achieving appropriate outcomes, and/or the college is 
managing well and improving the quality of its services for learners [but 
{summary of concerns).] 

OR 

• HMIE is not confident that .... College has effective learning and teaching 
processes and/or learners are achieving appropriate outcomes and/or the college is 
managing well and improving the quality of its services for learners. HMIE is 
concerned that {summary of major concerns). 


Review Phase Two: follow-through (differentiated) 

Differentiated follow through scenarios 

Differentiated reviews will be on a continuum from a very light touch through many 
gradations to a review similar in depth and breadth to the current model. The total 
number of reviewer days spent in a college implementing the proportionate review 
model will be determined accordingly. In those colleges where provisional statements 
of confidence following review phase one are most positive, follow-through will 
focus primarily on good practice. Provisional grades of fair or unsatisfactory 
awarded during review phase one will be investigated more fully as necessary at 
follow-through review. 

The following example scenarios are intended to be indicative rather than 
prescriptive. 

6.1 follow-through phase scenario where HMIE has provisionally expressed 

confidence but has identified concerns. 

The purpose of the follow-through phase is to probe further the major strengths and 
concerns identified in A5: Learning and teaching process and/or A7: Learner 
progress and outcomes and/or Leadership and quality > management with a focused 
evaluation. Reviewers will seek examples of good practice that can be written up for 
wider dissemination, and will also use selected A Elements to drill down to identify 
the causes and, where appropriate, the extent of strengths and weaknesses. Activities 
may include more observations of learning and teaching within the same subject areas 
as those covered in the core review. 


The size, composition and deployment of the review group will be determined by the 
agenda confirmed by HMIE following full discussion with the college at the end of 
Review Phase One. HMIE will take account of weaknesses identified during phase- 
one review, the size of the college, the diversity of its curriculum and the range of 
learning modes. In most cases, between two and eight reviewers will be involved for 
periods of two to three days each. 

6.2 Follow Through Phase scenario where HMIE has not been able to express 
provisional confidence after Review Phase One, and has identified major concerns 

The purpose in this instance will be to follow through comprehensively the strengths 
and major concerns identified in A5: Learning and teaching process, A7: Learner 
progress and outcomes, and Leadership and quality management, using all the 
A elements and B elements (as appropriate) to drill down, and applying at least A5 
and A7 to additional subject areas to identify the causes and extent of strengths and 
weaknesses. 

The size, composition and deployment of the review group camiot be predetermined. 
The scale of activity may require a group similar in size to the Review Phase One 
team, with individual reviewers being deployed for up to five days each. 

7. Appeals, concerns and complaints 

Colleges that believe reviewers have not received all available relevant evidence 
before arriving at an evaluation grade or confidence statement should submit any 
additional evidence in writing to the MI within five working days of the oral feedback 
by the review team. The MI, consulting members of the team as necessary, will 
consider any additional evidence and reply in writing, with a decision on any changes 
to be made.. 

College managers who have a concern or complaint about the conduct or the 
outcomes of a review should raise it in the first instance with the MI. 

If matters are not resolved to the satisfaction of the college, it should be raised with 
HM Assistant Chief Inspector responsible for the FE review programme who will 
seek to resolve the issue. 

If the appeal, concern or complaint is not satisfactorily resolved at this stage the 
college should pursue it through HMIE’s published complaints procedure. Colleges 
will receive printed copies of the procedure in advance of the review, and it is also 
available at www.hmie.gov.uk . 


8. Reports 

All reports will be in the public domain but will be prepared for different audiences: 


For college managers and staff 


• a practitioner report mainly in bullet point format, comprising evaluative text 
with main points for action (MPfA) and examples of good practice will be issued 
to the principal and subsequently made available on the HMIE website. The 
combined report on both phases, which will be written in technical language, may 
be between 7,000 and 19,000 words. 

For Scottish Ministers, SFEFC, FE College Boards of Management, the college ’s 
learners and the general public 

• a stakeholder report of up to 3,000 words in plain English will be published 
answering key questions based on the Confidence criteria. 


For prospective learners in the college 

• A report will be written by the college in plain English, moderated as necessary 
and approved by HMIE and published by the college. This report will be based on 
generic guidance to be provided by HMIE following consultation with colleges, 
SPARQS and the NUS. 


Timelines for the issue and publication of all the above reports will vary 
depending on FIMIE findings at the review phase one stage and on the nature and 
extent of the follow-through phase. 

9. Follow-up reviews 

On completion of phase one and the follow-through review phase, it will be for 
SFEFC to determine whether, when and how HMIE should conduct further follow-up 
review. SFEFC will wish college HMI to monitor college progress in main points for 
action and points for development in all colleges. It is expected that any college 
receiving a statement of “no confidence” from HMIE will be required to submit an 
action plan to the Council, and HMIE will be asked to conduct a follow-up review 
within timescales to be determined by the Council in discussion with HMIE. 

10. Encouraging the adoption of better practice across the FE sector 

Good practice will be: 

• provisionally identified by the college prior to review 

• provisionally identified by reviewers during Review Phase One 

• confirmed and further identified during follow-through reviews with the key 
managers and staff responsible 

• written up jointly by college managers, staff and reviewers. 

• Disseminated and promoted by HMIE and partner bodies. 
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Appendix 1 

The proportionate review process 




Confident Not confident 


1 



f 

Reports 

- stakeholder 

- practitioner 

- learners 


1 

Refer to SFEFC for nature and extent of 
follow-up 


Appendix 2 

Acronyms and abbreviations 


AA 

Associate Assessor 

AMI 

Assistant Managing Inspector 

FE 

Further Education 

FES 

Further Education Statistics 

HMI 

Her Majesty’s Inspector 

HMIE 

Her Majesty’s Inspectorate of Education 

KPI 

Key Performance Indicator 

MI 

Managing Inspector 

MPfA 

Main Points for Action 

NUS 

National Union of Students 

PI 

Performance Indicator 

PLP 

Personal Learning Plan 

QE 

Quality Element 

QI 

Quality Indicator 

SFEFC 

Scottish Further Education Funding Council 

SPARQS 

Student Participation in Quality Scotland 


